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Abstract. The g-character is a strong tool to study finite-dimensional 
representations of quantum afiine algebras. However, the explicit for- 
mula of the g-character of a given representation has not been known so 
far. Frenkel and Mukhin proposed the iterative algorithm which gener- 
ates the g-character of a given irreducible representation starting from 
its highest weight monomial. The algorithm is known to work for vari- 
ous classes of representations. In this note, however, we give an example 
in which the algorithm fails to generate the g-character. 



1. Background 

1.1. Finite-dimensional representations of quantum afRne algebras. 

Let g be a simple Lie algebra over C, and let Uq{Q) be the untwisted quantum 
affine algebra of q by Drinfeld and Jimbo [DH 102^ Ij]. 

The following are the most basic facts on the finite-dimensional represen- 
tations of Ugig), due to Chari-Pressley [UPniUP2] : 

(i) The isomorphism classes of the irreducible finite-dimensional represen- 
tations of Uq{Q) are parametrized by an n-tuple of polynomials of constant 
term 1, P = (Pi{u))i^j, where I = {1,... ,n} and n = rankg. The poly- 
nomials P are often called the Drinfeld polynomials because an analogous 
result for Yangian was obtained earlier by Drinfeld |D2| . 

(ii) For given Drinfeld polynomials P, let V(P) denote the corresponding 
irreducible representation. For a pair of Drinfeld polynomials P = {Pi[u))i^i 
and Q = {Qi{u))i^j, let PQ := {Pi{u)Qi{u))i^i . Then, y(PQ) is a subquo- 
tient of y(P) «)y(Q). 

(iii) A representation V^(P) is called the zth fundamental representation 
and denoted by K;, (a) if Pi{u) = 1 — au and Pj{u) = 1 for any j ^ i. 
Suppose that Drinfeld polynomials P are in the form 



(1.1) P,{u) = \[{l-afu). 

k=l 



Namely, a[!^ are the inverses of the zeros of Pi{u). Then, as a consequence 
of (ii), V(P) is a sub quotient of the tensor product of fundamental repre- 

sentations (g),^, (8)fcLi K.,(4'^). 
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For Q of type Ai, the structure of V(P) for an arbitrary P is known |CPlj . 
Also, when g is simply-laced, the relation between T^(P) and the so-called 
standard representations is described by an analogue of the Kazhdan-Lusztig 
polynomials jNlj . So far, no more general results are known for the structure 
of V{P). 

1.2. q-Character. To study the structure of V(P), the q-character Xg was 
introduced by Prenkel and Reshetikhin [FR ]. It is an injective ring homomor- 
phism from the Grothendieck ring of the finite-dimensional representations 
of Uq{Q) to the Laurent polynomial ring of infinitely -many variables ^^a? 
i G I,a eC^, 

(1-2) Xg ■■ Rep ^ nYi%']iei;aeCx ■ 

The variables Yi^a are regarded as affinizations of the formal exponentials 
exp(cjj) of the fundamental weights tUi of Uq^q). By replacing Yi^a with 
ex.p{u!i), reduces to the underlying C/g(g)-character of V(P) with 

respect to the standard embedding C/g(g) C Uq{g). 

There are several equivalent ways to define the g-character. 

(i) By universal R-matrix. This is the original definition of [FR] . The idea 
originates from the transfer matrix, which plays the central role in the quan- 
tum inverse scattering method, or the Bethe ansatz method for integrable spin 
chains such as the Heisenberg XXX model |TF] . The g-character Xq(y) of 
a representation V is defined as a partial trace of the universal i?-matrix of 
Ug{Q) on V. 

(ii) By weight decomposition. It is shown also in [FRj that Xqi^) is re- 
garded as the formal character of the weight decomposition of V with respect 
to certain elements in the Cartan subalgebra in the 'second realization' of 
Uqis) [D2j . Hernandez extended this definition of Xg to the affinizations of 
the full family of the quantum Kac- Moody algebras [H21 IH4j . 

(iii) By quiver varieties. When g is simply-laced, Nakajima [Nil IN2] 
geometrically defined a i-analogue of g-character Xq,t (the q,t- character) 
as the generating function of the Poincare polynomials of graded quiver 
varieties. Then, Xg is obtained by Xg = Xg,i- The algorithm of calculating 
Xg,t is given based on the analogue of the Kazhdan-Lusztig polynomials. 

(iv) By axiom. In |N2j the axiom which characterizes Xg,t in (iii) is given. 
The axiom is further extended for non simply-laced cases in [HI] . Then, Xg 
is obtained by Xg = Xg,i- 

Before the introduction of the g-character, the spectrum of the transfer 
matrix defined by the trace on a so-called Kirillov-Reshetikhin (KR) repre- 
sentation [KR| of Uq{Q) was extensively studied by the Bethe ansatz method 
f [Rn [B2l [Ml iBRl KH IKNSI iKNHl iKOSl iKSl ITK] . etc.). The fundamen- 
tal representations V^-(a), for example, are special cases of the KR repre- 
sentations. Because of Definition (i) above, these results, including many 
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conjectures, are naturally translated and restudied in the context of the q- 
character [FE ]lFMT|[CMlK()SYl lN3 llH3l[H6] . As a result, the g-characters 
of the KR representations are, not fully, but rather well understood now. 

However, beyond the KR representations, not much is known for the 
explicit formula of the g-character except for some partial results and con- 
jectures (e.g., (miNNT|lNN2l[NN3] V 

1.3. Frenkel-Mukhin algorithm. We say that a monomial in Z[YJ^^]jg7.agCx 
is dominant if it is a monomial of variables Yi^a, i I,a (z C^, i.e., without 
Yf^ . Suppose that Drinfeld polynomials P are in the form (jl.ip . Then, 
XqiV{P)) contains a dominant monomial 

rii 

(1.3) -+=nn^.a« 

iel k=l 

called the highest weight monomial of V^(P) |FRj . Since P and m+ are in 
one-to-one correspondence, we parametrize the irreducible representations 
of Uq{Q) by their highest weight monomials as V{m+), instead of V(P), from 
now on. 

Frenkel and Mukhin |FMlj introduced the iterative algorithm which gen- 
erates a polynomial, say, £ ^K^^]ie/;aecx from a given dominant 
monomial m_|_. We call it the FM algorithm here. A priori, it is not clear 
whether the algorithm does not fail (i.e., it is not halted halfway); also it is 
not clear whether it stops at finitely many steps. It was conjectured that 

Conjecture 1.1 ( |FM1] . Conjecture 5.8). For any dominant monomial m_|_, 
the algorithm never fails and stops after finitely many steps. Moreover, the 
result x{m+) equals to Xq{^{^+))- 

The algorithm is fairly practical so that, assuming the conjecture, one 
can explicitly calculate the g-characters of representations, by hand, or by 
computer, when the dimensions are small. 

Conjecture 11.11 is partially proved by |FMlj as we shall explain now. We 
say a representation V{m-^) is special if its highest weight monomial m+ 
is the unique dominant monomial occurring in Xg(^(w.+ )). For example, 
the fundamental representations are special [FMlj. More generally, the KR 
representations are special [N3l IH31 IH6] . (See |jH5j for further examples of 
special representations.) 

Theorem 1.2 ( |FM1| . Theorem 5.9). IfV{m^) is special, then Conjecture 
li.il is true. 

In particular, the FM algorithm is applicable to the fundamental rep- 
resentations and the KR representations, and provides the aforementioned 
results for their (/-characters. We note that there are also many nonspecial 
representations for which Conjecture 11.11 is true; e.g., g of type A2 with 
m+ = where F(m+) ~ ViYi^i) ® ViYi^iY,^q2). 
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The purpose of this note is to give a counterexample of Conjecture 11.11 
More precisely, it is an example where the algorithm fails in the sense of 
[FMT] (see Definition EZl). 

In Section 2 the FM algorithm is recalled. In Section 3, as a warmup, we 
give two examples in which the algorithm works well. Then, a counterex- 
ample is given in Section 4. Taking this opportunity, we also demonstrate 
the synthesis of the FM algorithm and Young tableaux in \BR\ IKOSj IKS^ 
iNTl [NNT| lNN2l lNN3] by these examples. 

We thank D. Hernandez for his helpful comments. 

2. FM ALGORITHM 

Here we recall the FM algorithm. The presentation here is minimal to 
describe the counterexample in Section 4. We faithfully follow [FMl^ Section 
5.5], so that the reader is asked to consult it for more details. 

2.1. Preliminary: g-character of Uq{5l2)- The FM algorithm is based on 
the explicit formula of the g-characters of the irreducible representations of 

Uq{si2) [GPU EE]. 

Example 2.1. Let Wr{a) be the irreducible representation Uq{5l2) with 
highest weight monomial 

r 

(2.1) m+ = JJy„g.-2fc+i, 

k=l 

where we set Ya := Yi^a- Then, its g-character is given by 

r i 

(2.2) Xq{Wr{a)) = m^Y.Il^aqr-2,+^^ := Y^q-.Y^q. 

i=0 j=l 

Generally, the g-character of any irreducible representation of f7g(sl2) is 
given by a product of (12. 2p as follows [CPlj : Let S^^^ be the set of the 
indices of the variables Y;, in ()2.ip . i.e., Sq^,. = {aq''~'^^^^}k=i,...,r ■ We call it 
a q-string. We say that two g-strings Tia,r and Sa',r' are in general position 
if either (i) the union Tia^r U '^a'y is not a g-string, or (ii) Sa,r C '^a'y or 
Sa'.r' C T,a,r- Then, 

Example 2.2. Let m+ G ^Y^^] a^c^ be a given dominant monomial. 
Then, one can uniquely (up to permutations) factorize m_|_ as 



(2.3) 



-+=fi( n 
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where J^ai^n,- ■ ■ , ^a^^rh ^-re g-strings which are pairwise in general position. 
The g-character of V{mj^) is given by 

k 

(2.4) x,{V{m+)) = J{x,{WrM^))■ 

i=l 

2.2. Algorithm. Let us start from some key definitions. 

Definition 2.3. (i) We say that a monomial m € ^[^j^^]je/;aecx is i- 
dominant if it does not contain variables , a G C ^ . 

(ii) For a polynomial x G ^[^i^^]jG/;aeC>< ^ monomial m occurring 
in X with coefficient s, a coloring of m is a set of integers such that 
< Si < s. We say that a polynomial x is colored if all monomials occurring 
in X have colorings. 

(iii) Let ^[^j^^]je/;aGC>< be a colored polynomial, and let m be a monomial 
occurring in x with coefficient s G Z>o and coloring {sjjjg/. We say that m 
is admissible if, for any i G / such that Si < s, m is i-dominant. 



Let 



(2.5) 



j:Cj^ = -l 



where Cij = 2(aj, aj)/(aj, a^) is the Cartan matrix of g. The monomials 
Ai^a are regarded as affinizations of the formal exponentials exp(Q;i) of the 
simple roots Oj of Ug{g). 

The FM algorithm is an iterative algorithm, and its main routine utilizes 
the following procedure called the i- expansion: 

Definition 2.4. Let i € /, x be a colored polynomial, and m be an admis- 
sible monomial occurring in x with coefficient s and coloring {sj}jg/. Then, 
a new colored polynomial imix)i called the i-expansion of x "with respect to 
m, is defined as follows: 

(i) If Si = s, then im{x) = X- 

(ii) If Sj < s, we define im{x) in the following two steps. 

First, we obtain a colored polynomial ^ which depends on m and i (but 
not on x) as follows: Let m be the ith. projection of m, i.e., Yia — ^a'^ 
and = 1 for any j ^ i. Let XqA^^)) = rn{\ + X^p-^p) be the q- 

character of the irreducible representation Vijri) of Uq^{5{2) with highest 
weight monomial m, where Mp is a product of A^^ (see (j2.2p . (j2.4p . and 



(1231) 1. Then, 

(2.6) ^ = m(l + ^Afp), 
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where Mp is obtained from Mp by replacing all A^ ,^ by A^^ in (j2.5p . 

Next, we obtain im{x) by adding the monomials occurring in /i to x as 
follows: Suppose that a monomial n occurs in ^ with coefficient t. If n does 
not occur in Xi we add n to x with coefficient t{s — Si) and set its coloring 
{s^}jg7 as = for any j ^ i and «■ = t(s — Si). If n occurs in x with 
coefficient r and coloring {rjjjg/, we set the coefficient s' and the coloring 
{s'j]jel of n in im(x) as s' = max{r,ri + t{s - Sj)}, s^- = rj for any j / i, 
and = + — Sj). The coefficients and the colorings of other monomials 
occurring in x are unchanged in im{x)- 

Note that the i-expansion is defined only if m is admissible. 

Remark 2.5. In (j2.6p . the coefficient of m in /x is always 1. Therefore, both 
the coefficient and the ith coloring of m in im{x) are s in Definition I2.4[ In 
other words, the z-expansion of x with respect to m is designed to saturate 
the rth coloring of m to its coefficient. 

Definition 2.6. (i) The Uq{Q)-weight of a monomial 

r=l s=l / 

is defined by Yli&ii^i - 

(ii) We equip the C/g(5)-weight lattice P := ^^^jZuji with a partial order 
such that A > A' if A — A' = Ojaj, G Z>o, and call it the natural partial 
order in P. 

Now let us define the FM algorithm. It is an algorithm generating a col- 
ored polynomial x("T'+) ^ '^[^{^hel-aec^ from a given dominant monomial 

Definition 2.7 (The FM algorithm). Let m+ be a given dominant mono- 
mial in Z[^j^^]ie/;aGCx 1 and A+ be the C/q(0)-weight of m+. Choose any 
total order in the set P<\^ := {/i € P | /x < A+} such that it is compatible 
with the natural partial order in P; then enumerate the elements in P<\_^_ 
as Ai = A+ > A2 > A3 > 

Step 1. We set the colored polynomial X by x = with the ith coloring 
of m+ being for any i G I. 

Step 2. Repeat the following steps (i)-(iii) for A = Ai, A2, A3, . . . . 

(i) Let X be the colored polynomial obtained in the previous step. Let 
mi, . . . , m-t be all the monomials occurring in x whose C/q(0)-weights 
are A. If there is at least one non- admissible monomial among them, 
then the algorithm halted halfway. We say that the algorithm fails 
at rrii if rrii is one of such non- admissible monomials. 

(ii) Repeat the following for all i G / and all k G 1, . . . ,i: Replace x 
with the i-expansion imkix) of x with respect to ruk- 
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(iii) If there is no monomial occurring in x whose ?7|j(0)-weight is less 
than A in the total order of P<a+j then set = X a-iid the 

algorithm stops (i.e., completes). 

It follows from Remark 12.51 that, if the algorithm successfully stops, the 
ith coloring of any monomial m occurring in x(m4_) equals to the coefficient 
of m for any i ^ I. Thus, once x(^+) is obtained, one can safely forget the 
coloring. 

3. Examples 

Let us see how the FM algorithm works in good situations. This is a 
warmup to understand the 'bad situation' in the next section. 

3.1. Example 1. Let us consider the case where g is of type A2 and the 
representation l/(m+) has the highest weight monomial 

(3.1) m+ = Yiq2Y2^q-i. 

The C/g(f|)- weight of m+ is A+ =001 + 002- It is well known that V{mj^) is 
an evaluation representation of the adjoint representation V^^+oja of Uq{Q). 
As a C/g(g)-representation, it is isomorphic to K)i+aj2- It is also known that 
y(m+) is special [HSJ so that the FM algorithm is applicable. 

We use the following data: (?i = (?2 = 9, "i = 2lji — u;2, 02 = —^i + 2tJ2, 
and 

Now let US execute the FM algorithm step by step. We choose a total 
order in -P<a+ as 

Ai = A+, A2 = A+ — ai, A3 = A+ — 02, A4 = A4. — 2ai, 
(3.3) A5 = A+-ai-a2, Ae = A+ - 2a2, A7 = A+ - 2ai - 02, 
Ag = A+ — «! — 2a2, Ag = A-i- — 2ai — 2a2, ■ ■ ■ , 

where the rest of the order is irrelevant. 

Step 1. Set X = = ^i,g2l2,i}-i with the coloring of being (0,0). 
Step 2. (1) A = Ai = wi + 002'. 

The 1-expansion of x with respect to m-|_ = Yi q2Y2^q-i is done as follows: 
Since Yi q2Y2^q-\ = Yg2, we have 

Xq{V)=Yq2il + A^'qs), 

1^ = ^2,9-1 (1 + ^^^3) = n,g2>^2,,-i + ^i7g4^2,g-i5^2,g3 , 

Im+(X) = ^l,g2^2,g-i +^17,^^2,9-1^2,93, 

(1,0) (1,0) 

where (1,0) represents the coloring. Then, x is replaced with lm+(x)- 
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Similarly, the 2-expansion of x with respect to m+ is calculated as 

(1,1) (1,0) (0,1) 

(2) A = A2 = —loi + 2lo2- Prom now on, we only write down the nontrivial 

z-expansions, i.e., the cases where .Sj < s. 
The 2-expansion w.r.t. 1^^^^412,5-1^2,93: 

^' = n:,' i^2,,-i>2,,3(i + ^2,i)(i + ^2,^4) 

= n:,4i^2,,-i>2,,3 + yi,ii;-\i^2:,'i2,,3 + Y,^^-.Y-^, + Y^,,Y,-x:^„ 

X = n,,^i2,,-i + y,-\r2,,-i^2,,3 + yi,iyi,,2y2-:,' + Y-i,iF-\y2-:,'^2,,3 

(1,1) (1,1) (0,1) (0,1) 

+Y2,,-^Y,:i.+Y^,^Y-^%-^,. 

(0,1) (0,1) 

(3) A = A3 = 2u!i - L02- 

The 1-expansion w.r.t. Y\^\Yx^q2Y^^: 

= Y,,yY^,^.Y^l{\ + A-\, + ^i;jA-;3) 

= ^l,l^l,q2^27q^ + ^I,l^l7q4^27q ^2,g3 + ^i7g2^l7g4^2,g3 , 

X = n,,2^2,,-l + yi-\y2,,-l>^2,,3 + 11,117,52^27,' + ^l,in7,4^275'^2,53 
(1,1) (1,1) (LI) (1,1) 

+ ^2,,-i>^27^5 + y^^y^lY^^. + y-^,y-^^:y2,,^. 

(0,1) (0,1) (1,0) 

(4) A = A4 = + Zuj2- No nontrivial z-expansions. 

(5) A = As = 0. 

The 1-expansion w.r.t. Y2^q-iY~\^: 

fJ- = ^2,q-l5^2,q5> 

X = yi,A,-^ + y^,^.y2,,-^y2,,^ + ^,117,5^^^2^ + ^,1^7- 4i^2:5'^2,53 
(1,1) (1,1) (1,1) (1,1) 
+ ^2,5-^2:,' + ^i.i^2:,'^27,« + n:,' n;,' i^2,,3. 

(1,1) (0,1) (1,0) 

(6) A = Ae = 2>u}i — 3^2- No nontrivial i-expansions. 

(7) A = A7 = —2ui + U2- 



ON FRENKEL-MUKHIN ALGORITHM 



9 




(1,1) (1,1) (1,1) (1,1) 



~h Yn l Yr. r: H" Y\ 1 Yr, ^ 1^ tz Y^ oY^ aYo ^3 ~|~ Y. oY^, r. . 




X = Y^,,2Y,^g-^ + y-^iy2,,-i>^2,,3 + ii.iii.g^y^:/ + yi,iyi-\y2-'i^2,,3 



(9) A = Ag = —uJi — UJ2- There is no nontrivial i-expansions; furthermore, 
there is no monomial occurring in x in P-4p whose C/g(g)-weight is less than 
Ag in the total order. Therefore, we set x("i+) to be x iii p.4p . and the 
algorithm stops. 

Thus, we obtain the ^-character of V{m^) as x p.4p by forgetting 
coloring. 

Next, let us introduce a diagrammatic notation of monomials by Young 
tableaux, following (BH IKOSI IKSI INTI IFRI [FM2l iNNl] . 

To each letter a = 1,2,3 within a box of a Young diagram Y, we assign 
a monomial as (cf. |FR1 Section 5.4.1]) 



where the subscription 'ij' indicates that the box is located in the ith row 
and jth column of Y. To each tableau T on Y, we assign a monomial m{T) 
by multiplying the monomials assigned to all the boxes in T. For example, 
the first and the second monomials in x in (|3-4|1 are represented by tableaux 
as 



(3.4) 



(1,1) (1,1) (1,1) (1,1) 

+ Y,,,-.Y-^. + Y,,,Y,-^Y-l, + y,^^.y,-ly2.,3 + y-^^y^-i 
(1,1) (1,1) (1,1) (1,1) 



(3.5) 




(3.7) 



(3.6) 




^1^^4^2,g-i^2,g3- 
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Figure 1 . The flow of the FM algorithm by Young tableaux 
for Example 1. The symbol i,q^ at an arrow represents the 
action of . The suffix i at a tableau indicates that s,- < s 
when X is to be i-expanded with respect to the corresponding 
monomial. 



The definition (j3.5p is designed so that the following equalities hold |FRt 
Section 5.4.1]: 

(3.8) [T]^= 0^.^., A 



2,g 



-2i+2j+2 



Namely, the multiplication of A^^ is regarded as the 'action' of changing 
the letter i to z + 1 in a tableau, if a is appropriately chosen. 

With this notation, one can concisely keep track and express the whole 
process of the algorithm presented above by the semistandard tableaux of 
shape (2, 1) as in Figured! Moreover, as a corollary of Figured! we obtain 
the tableaux expression of the Q'-character 

(3.9) XgiVim+))= Yl "^(^)' 

TGSST{2,1) 

where SST(2, 1) is the set of all the semistandard tableaux of shape (2, 1). 



Remark 3.1. For g of classical type, similar tableaux expressions to ()3.9p 
have been conjectured and partially proved for a large class of irreducible 
representations V{X/iJ,) parametrized by skew Young diagrams X/fi [BR] 
IkHS! Iks! IfH IFM2I InFT! Wm\ [NN3! IHS] . More precisely, there is a 
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tableaux expression for the 'Jacobi-Trudi-type determinant' which 
Ues in the image of the homomorphism Xq- For types An and it is known 
that xi^/f^) = Xgi^i'^/ f^)) for any skew Young diagram |H5l IH7] . For type 
An, this was also shown in the context of the character of Yangian y(0[„) 
[NT] . For types C„ and Dn [NNTI lNN2l lNN3] . for a (non-skew) Young di- 
agram A, it was conjectured that xW = Xqi^W)- In general, xi^/f^) is 
conjectured to be the g-character of, not V{X/n) itself, but some represen- 
tation which has V{X/fi) as a subquotient. Using this opportunity, let us 
withdraw our false claim for types C„ and Dn in [NN1[ INN21 INN3| that we 
expect that x(A///) = Xqi^i^/ ^/^^/m "i-s connected. A counterexample is 
given by g of type C2 with A = (2, 2, 1), /i = (1). 

Remark 3.2. The underlying [/g(0)-character of y(m+) is symmetric under 
the Dynkin diagram automorphism 1 <^ 2. However, we see in Figure [J that 
the ?7q(g)-structure of F(m_|_) is not so. Of course, this is not a contradiction, 
because the highest weight monomial m+ in (j3.ip are not symmetric under 
the automorphism. 

3.2. Example 2. To convince the reader further that the FM algorithm is 
well designed, let us give another, a little more complicated example, where 
the coefficients of some monomials in the g-character are greater than one. 
We consider the case where g is of type C2 and the representation y(m_|_) 
has the highest weight monomial 

(3.10) m+ = Y2^g-iY2,g. 

The ?7q(g)-weight of m+ is A+ = 2iJ2- We faithfully follow the convention 
in [FRt IFMT] : in particular, 02 is the long root. 

Since any monomial occurring in Xqi^ (rri-^-)) for (j3.10p should occur in 
the product Xg(^(^2,<7-i))XQ(^(^2,g)), and 

(3.11) XqiV{Y2^,-i)) = Y2,g^i + Yi,,Y,^g2Y^-^\ + Y,,iY;^^\ 

+ ^i7f/2^i7g*-^2,g + Y-^„ 

(3.12) Xq{V{Y2,g)) = Y2,g + Y,^^2Y,^^,Y-^\ + Y.^^.Y'^l 

one can immediately see that is the only possible dominant monomial 
in Xq(Y{m+)). Thus, V{m^) is special, and the FM algorithm is applicable. 
It also implies that ^(^2,9-1) ® ^(^2,<j) is irreducible and isomorphic to 
F(m+). In particular, as a ?7q(0)-representation, V{m+) is decomposed as 
K)2 K;2 — ^21.^2 ® ^20^1 © Yq with dimension 5x5 = 14 -|- 10-1-1, and its 
C/q(0)-weight diagram is given in Figure [2j 

Keep in mind that (Definition 12.41 (ii) ) the i-expansion should be done, not 
with Uq{sl2), but with Uq-{5l2)- Then, the algorithm can be straightforwardly 
executed with the data: qi = q, q2 = , and 

(3.13) ^1 = ^i^ag-i^l,ag^2,a, ^2,a = ^2,aq-'2^2,aq^^^,aq-^^^,a.g- 
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1 2UJ2 



2 2 2 
2 5 — 2 



2 2 2 
1 

Figure 2. The f7q(g)-weight diagram of V{mj^) in Example 
2. The numbers represent the weight multiphcities. 



Again, the flow of the algorithm can be expressed with tableaux of shape 
(2, 2). We assign a monomial to each letter a = 1, 2, 2, 1 within the box at 
position (i, j) as (cf. [FRj Section 5.4.3]). 

-1 



(3.14) 



. . = ^l,g-2i+2i , 



y: 



I „-2i+2j+2^2,g-2»+2i+l , 



1 



1 



I „~2i+2j+6- 



For example, the highest weight monomial m_(_ is represented as 
(3.15) 



m 



1 


1 


2 


2 



The 'action' of A^^ on a box is given by 



(3.16) 



S, 



^l^q-2i+2j + l 



4-1 



^l_g-2i+2j+5 



1 



Then, the flow and the result of the algorithm is expressed by tableaux in 
Figure [3l 

We note that two monomials occur in Xq(y(m_|_)) with coefficient two, 
and, in Figure [3l each monomial is represented by two different tableaux 
such as 

(3.17) 



m 



m 



m 



1 


1 


T 


T 



m 



1 


2 


2 


T 



Purely from the point of view of the FM algorithm, this is redundant, be- 
cause the FM algorithm does not distinguish tableaux if they represents the 
same monomial. However, by doing this, we have the following tableaux 
expression of the g-character 



(3.18) 



Xq{V{'m+)) = ^ m(T), 



TgTab 



where Tab is the set of the tableaux occurring in Figure [31 Remarkably, 
the formula (I3.18P exactly coincides with the tableaux expression in [NNH 
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1 


2 




2 


2 


T 


1 


2 


1 


1 


2 












1,9' 


2 


2 










2 


1 


2 





2,(75 2,g 



Figure 3. The flow of the FM algorithm by Young tableaux 
for Example 2. The equality between two tableaux means 
that they represent the same monomial. 



INN3] based on the Jacobi-Trudi-type determinant, thereby showing nice 
compatibility between two approaches. 

Remark 3.3. The implementation of the FM algorithm with tableaux (or, 
equivalently, with paths) of [NNl^ INN3 ] demonstrated here, can be general- 
ized to the skew diagram representations of type Cn |NN4| . See also Remark 
01 
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4. Counterexample 

Now we are ready to present an example where the FM algorithm fails in 
the sense of Step 2 (i) of Definition 12.71 

We consider the case where g is of type C3 and the representation l/(m+) 
has the highest weight monomial 



(4.1) 



>"l,g4y2,g>3,q-2- 



According to |NNH Conjecture 2.2, Theorem A.l], it is expected to be de- 
composed into \{oi+uj2+'^s 

representation, with dimension 512 + 189 + 90 + 70 + 21 + 14 = 896. The 
algorithm is executed with the data: qi = q2 = Q, Qs = , and 



(4.2) 



^3,1 = ^3,ag-2^3,ag2^2,ag-l^2,ag- 



Again, the process of the algorithm can be expressed by Young tableaux 
of shape (3,2,1). We assign a monomial to each letter a = 1,2,3,3,2,1 
within the box at position as 



(4.3) I 2 







i + 2j , 


ij 






y-1 






y-1 
2,g-2 


i+2j+3^3,(?-2»+2i+2, 





- ^2,g-2 


ij 










1,9 



^1 



3 g-2i+2i+6 ) 



For example, the highest weight monomial m+ is represented as 



m 



(4.4) 



1 


1 


1 


2 


2 




3 







yi,iyi,,2yi,,4(yi;,iy2,g-0(n:g2i^2,5)(^2;/-i^3,g-2) 
yi,q4y2,gy3,q-2. 



The 'action' of A^^ on a box is given by 



(4.5) 



0; 



^2.q'2i+2j+2 



\ -2i+2j+6 



^3^q-2i+2j+4 



^1 -2i+2j + 7 



Theorem 4.1. T/ie FM algorithm fails for m+ in 
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1 


1 


1 


m+ — 


2 


2 






3 


1,2,3 



3,1. 





1 


1 


1 


1 


1 


1 


mi — 


2 


2 


me = 


2 


3 






3 


1,2 


3 


1,3 


2,q^ 






^,9'^^^S<^3 


1 







1 


1 


1 




1 


1 


1 




2 


3 






2 


2 






3 


1,3 




2 


1,3 





1 


1 


1 




1 


1 


1 


ms = 


2 


3 




ma = 


2 


2 






2 


1,3 




1 









1 


1 


1 




2 


3 






1 


2,3 



Figure 4. The diagram explaining how the FM algorithm 
fails for m+ in (j4.ip . 



Let us prove the theorem. We set 



(4.6) 


mi 




(4.7) 


1712 


:= A-^^,A,^,m^ = {Yi^g.Yi^^,)iY^^^-^Y2,gY^-^,), 


(4.8) 


1713 


:= A-2,^3":l"^+ = (n',2n,,4)(r2,,-i>^2j)^3,,2 


(4.9) 


171/1 


•= ^i^2;j2^3^1"^+ = ^,,2(^2,9-1 5^27,3)^3,92, 


(4.10) 


ms 




(4.11) 


me 


:= = {Yi^g2Yi^g4){Y3^g^2Ys^^2). 



We show below that the algorithm fails at m^. See Figure H] for the outline 
of the proof in terms of tableaux. 

Lemma 4.2. The monomial m^ occurs in x o-i some step in the algorithm. 



Proof. By (14. Ih . the 3-expansion of x with respect to m+ gives ^ = m+(l + 
A3 I), where // is the polynomial in (12. 6|) . Therefore, m-i occurs in x after 
the expansion. Next, by (I4.6p . mi is admissible, and the 2-expansion of x 
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with respect to mi gives = mi(l + A^^2 + ^2 1^2 + ^2 ^2)- Thus, 
occurs in x after the expansion. Finally, by (j4.8[) . is admissible, and the 
1-expansion of x with respect to gives /i = mi(l-|-^~^5 -|-^~^3A~^5)(1 -|- 
^"^3). In particular, 1714 occurs in x after the expansion. □ 

Let A (= uJi + UJ3) denote the C/g(g)-weight of m^. Let us show that the 
monomial is not admissible when x is going to be expanded at A; hence, 
the algorithm fails at m4. To see it, suppose that m4 is admissible when 
X is going to be expanded x at A. Since m4 is not 2-dominant, it should 
occur in the 2-expansion with respect to some 2-dominant monomial, say, n 
whose [/g(0)-weight is greater than A. Since {^i,a}ie/;aecx algebraically 
independent, n should be either = A2_g2m4 or m'^ = A^^2fni- Then, 
one can easily check that the 2-expansion with respect to generates m4, 
while the 2-expansion with respect to m'^ does not so. Therefore, n = m^. 
However, 

Lemma 4.3. The monomial ms does not occur in x f^t any step in the 
algorithm. 

Proof. By (j4.10p . there are six possible routes to obtain from m+ by 
i-expansions: (The symbol - — > represents the action of AT^,^.) 

(i) m_|_ * * ms. The 1-expansion of x with respect to m+ 
gives // = m-|_(l -|- A~K^). So, it does not happen. 

(ii) m_|_ * * ms. By the same reason as above, it does not 
happen. 

(iii) m+ — ^ rriQ * -^—^ m^. The 2-expansion of x with respect to m-|_ 
gives /X = m_|_(l + A"^^)- So, mg occurs in x- Then, the 1-expansion of x 
with respect to mg gives /x = m6(l + A~\, + A'^^^^A^K,). So, it does not 
happen. 

2,q^ 3,1 l,q^ 

(iv) ruj^ me m2 ^— > m^. The 3-expansion of x with respect to 
mg gives /i = m6(l + ^3 ^4 + ^3 1^3 ^4)- So, it does not happen. 

(v) m-|_ mi * ms. The 1-expansion of x with respect to mi 
gives /X = mi(l -|- A'T^^r^). So, it does not happen. 

(vi) m+ mi m2 m^. The 1-expansion of x with respect to 
m2 gives /x = m2(l + ^^gS ~'~ "^iq^^iq^"^' does not happen. 

Therefore, does not occur in x at any step. □ 

This completes the proof of Theorem 14. li 

Shortly speaking, the algorithm fails because it fails to generate ms which 
is an extra dominant monomial in Xg(^(?Tx+)). 

It is not difficult to find some other examples where similar phenomena 
happen. For example, it is a good exercise to check that, if g is of type D4 
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and the representation has the highest weight monomial 



(4.12) 




^l,g2l3,q-2^4,g-2, 



the FM algorithm fails at the monomial 



(4.13) 




where we use the diagrammatic notation in [FRt INN1[ INN2j . 

We conclude with a remark on a modification of the FM algorithm. 

Remark 4.4. Actually, in the counterexample above, the FM algorithm 
almost works except for missing one monomial m^. It suggests the follow- 
ing modification of the algorithm: when we encounter the non-admissible 
monomial m4 in the algorithm, one simply adds (the '2-ancestor' of 
771,4) to X with coloring (0,0,0), then restart the expansions from A = 2uj2- 
Then, we have checked by computer that the modified algorithm stops and 
certainly generates monomials represented by 896 tableaux as expected in 
|NNl[|NN3j . For general representations, this trace-back procedure is, a pri- 
ori, not well-defined, because one cannot uniquely determine the 'z-ancestor' 
of a given monomial. However, for the family of the skew diagram repre- 
sentations of type Cn in [NNTI INN3] . one can do so with help of tableaux 
representation (or, more conveniently, paths representation) of monomials. 
Observe Figure H] as a simple example. By modifying the FM algorithm 
with the trace-back procedure, we expect that Conjecture 11.11 is true for 
these representations, and it is supported by our computer experiment. The 
detail will be published elsewhere [NN4] . 
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